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Summary 


The 2005 Public Service Employment Act (PSEA), as well as the related appointment policies, 
provides deputy heads with the necessary flexibility to meet the needs of their organization, 
notably the choice of appointment process. The core values of merit and non-partisanship, 

as well as the guiding values of fairness, transparency, access and representativeness, will influence 
the choice of appointment process. 


Non-advertised appointment processes have been assessed by the Public Service Commission (PSC) 
as a risk area which needs to be more closely monitored by deputy heads to ensure respect of the 
core and guiding values. Further, the PSC has expressed a need to collect more rigorous and detailed 
information about non-advertised appointment processes. 


The ratio of non-advertised appointment processes to total appointment processes is used by the 
PSC as an indicator of organizational effectiveness and adherence to the guiding value of access. 
However, with incomplete information about the circumstances under which non-advertised 
appointment processes occurred, the ability to assess the risks associated with such appointment 
processes is limited. 


The purpose of the study was to describe data collection practices related to non-advertised 
appointment processes and the limits of the data available, as well as to explore associated risks 
and areas for improvement for the PSC and organizations. 


The study focused largely on the non-advertised appointment activities of four organizations: 
the Canadian International Development Agency, Citizenship and Immigration Canada, Library 
and Archives Canada and Transport Canada. It included approximately 50 files from each of the 
selected organizations, consisting of 10 advertised appointment processes and 40 non-advertised 
appointment processes for the period of April 1, 2007, to March 31, 2008. 


While all four of the organizations examined had a formal human resources (HR) system, 

three of the organizations did not rely solely on their HR system to provide information to the PSC. 
The Canadian International Development Agency, for example, maintains a separate Excel-based 
database to help manage the hiring process. In the case of Citizenship and Immigration Canada, 

data entry into PeopleSoft is highly decentralized, requiring the organization to cross-reference data 
in PeopleSoft with data from an Excel-based system. At Library and Archives Canada, the current 
version of the Human Resources Information System (HRIS) does not allow for customized queries; 
therefore, staff rely on the process number or on a manual review of the appointment file to determine 
the type of appointment process. Transport Canada is the only organization that uses its HR system 
to manage the hiring process as well as position information. 
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While the calculation of the leading indicators related to access is easy to understand and is well 
documented, organizations were unanimous in their concern that the ratios of non-advertised 
appointments (ACC-1 and ACC-3) do not provide a meaningful measure of the value of access 

to public service positions. (Refer to Appendix 3 for a complete list of the leading indicators related 
to effectiveness and adherence to the staffing values.) 


The study found that accuracy varied among organizations and that issues of both over-coverage 
and under-coverage exist. Errors in coding were also found. The amount of time and effort required 
for submitting data to the PSC and the capacity to respond to changing requests vary greatly 
among organizations. 


There is a limited level of coherence of information reported to the PSC for extensions to acting 
appointments. One organization treats such extensions to acting appointments as subsequent 
non-advertised appointments, while the remaining organizations treat such extensions as administrative 
measures. However, for the purpose of the quarterly reports to the PSC, two organizations consider 
these as appointments. 


The PSC is committed to clarifying the policy interpretation surrounding extensions to acting 
appointments. Further, the PSC will explore other options and methods of data collection 
from organizations. 
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Introduction 


Background 


The 2005 Public Service Employment Act (PSEA), as well as the related appointment policies, 
provide deputy heads with the necessary flexibility to meet the needs of their organization, 
notably the choice of appointment process. A process can be internal or external advertised, 
or internal or external non-advertised. The core values of merit and non-partisanship, as well 
as the guiding values of fairness, transparency, access and representativeness, will influence 
the choice of appointment process. 


As per the Public Service Commission (PSC) Policy on Choice of Appointment Process and related 
documents, due consideration should be given not only to the core and guiding values and to human 
resources (HR) planning but also to the position requirements, operational realities, workforce 
characteristics and other considerations related to short-term situations. However, the PSC 
considers that the core and guiding values should lead to advertised processes more often than 

to non-advertised processes. 


As part of their accountability, deputy heads are required to establish organizational criteria to 
identify the circumstances under which the use of non-advertised processes is appropriate for 
their organization. As per the Public Service Commission 2007-2008 Annual Report, non-advertised 
appointment processes may be appropriate under certain conditions and would likely be 

selected by the manager for appointments following a reclassification or promotions within 
developmental programs. 


Issue of study 


Over the years, non-advertised appointment processes have been assessed by the PSC as a risk area 
which needs to be more closely monitored by deputy heads to ensure respect of the core and guiding 
values. Therefore, the PSC has expressed a need to collect more rigorous and detailed information 
about non-advertised appointment processes. There is also a requirement for continuous monitoring 
to ensure a reliable information base for the assessment of organizational performance and respect 
of the core and guiding values. 


As noted in its annual report for 2007-2008, the PSC estimates that 43% of all appointments, 
internal and external, made under the PSEA in 2007-2008 were done through non-advertised 
processes. This figure is at 28% for external appointments. It is worth noting that for acting 
appointments only, 70% were made through non-advertised processes. 
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The PSC’s 2006 Report on Preliminary Data Collection Activities Related to Advertised versus 
Non-advertised Appointments identified problems with the data quality of appointment 
information reported by organizations. 


There are various means by which organizations can provide the information requested by the PSC. 
They have the option of supplying the information through a download from their HR system, 
or manually reporting the information using a PSC Excel template or other format. 


As indicated in the report by the Council of Systems Cluster Groups entitled A Retrospective and 
Current Analysis of the Corporate Administrative Systems Clusters in the Government of Canada 
2007-2008, a variety of administrative systems is available to public service organizations for the 
management of HR information. The two most common systems include the Government of 
Canada Human Resources Management System (PeopleSoft) and the Human Resources Information 
System (HRIS); however, there is an array of other HR systems, including, Transport Integrated 
Personnel System (TIPS) and Systems Applications Products (SAP). 


Currently, organizations must submit data to the PSC quarterly, or bi-annually for small organizations, 
on the use of advertised and non-advertised appointment processes, indicating as well whether the 
process was internal or external. With this information, the PSC calculates the ratio of external 
non-advertised appointment processes to total appointment processes, which is referred to as 

the Access | leading indicator (ACC-1); and the ratio of internal non-advertised appointment 
processes to the total appointment processes, which is referred to as the ACC-3 leading indicator. 
(Refer to Appendix 3 for a complete list of the leading indicators related to effectiveness and 
adherence to the staffing values.) The PSC uses the ratio of non-advertised appointment processes 

to total appointment processes as an indicator of organizational effectiveness and adherence to the 
guiding value of access. However, with incomplete information about the circumstances under which 
non-advertised appointment processes occurred, the ability to assess the risks associated with such 
appointment processes is limited. 
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Focus of the study 


Approach 


The purpose of the study was to describe data collection practices related to non-advertised 
appointment processes and the limitations of the data available, as well as to explore associated 
risks and areas for improvement for the Public Service Commission (PSC) and organizations. 


This study was conducted using Statistics Canada’s Quality Assurance Framework, which defines 
the six dimensions of data quality: relevance, accuracy, timeliness, accessibility, interpretability 
and coherence. A full definition of each of these dimensions can be found in Appendix 2. 


The study focused largely on the non-advertised appointment activities of four organizations: 
the Canadian International Development Agency, Citizenship and Immigration Canada, 
Library and Archives Canada and Transport Canada. The organizations selected are 

subject to the PSEA. 


Exhibit 1: Organization profiles 


Canadian Citizenship and 

International Immiaration Library and Archives 

Development Agency | Canada Canada Transport Canada 
Size Medium Large Medium Large 
Regions Ye 


No 
HR system used PeopleSoft HRIS 


Secondary tracking 
system used 


Criteria for 

non-advertised 

appointment Yes Yes Yes Yes 

processes Cal 
DSAR rating for Opportunity for 

ACC-1 improvement Acceptable Attention required | Acceptable 
DSAR rating for Opportunity for 
ACC-3 Acceptable Acceptable Acceptable improvement 


Reporting to PSC Quarterly 


General approach Use of secondary 
to extract data for tracking system and| File review based 
quarterly reports HR system on process number 


Use of secondary 


tracking system Use of HR system 


Source: Audit, Evaluation and Studies Branch, Public Service Commission of Canada 
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The study activities included interviews with key stakeholders from the four organizations in order 
to obtain an understanding of their data collection process and related risks. Interviews were also 
conducted with PSC stakeholders to identify the organizations’ reporting requirements and key 
reporting issues. The study also included an analysis of appointment and system documentation 

as well as other information, including organizational monitoring reports. Finally, the study included 
observation of the data collection process in the organizational human resources (HR) systems and 
the file review of a random sample of appointment processes. 


The study included approximately 50 files from each of the selected organizations, consisting 

of 10 advertised appointment processes and 40 non-advertised appointment processes for the 
period of April 1, 2007, to March 31, 2008. This sample allowed for an assessment of general data 
quality for each organization for non-advertised processes; the combined advertised processes from 
each organization allowed for an estimation of general data quality across all four organizations. 


For more information regarding the study objectives and methodology, see Appendix 1. 


Context 


There are various reasons for not soliciting applications for a process and for proceeding with 

a non-advertised appointment process. For example, for developmental programs, potential 
participants must meet the conditions of participation and the required qualifications, as defined 
in the organizational entry requirements. The subsequent non-advertised promotions are based 
on the participant meeting the requirements of his/her current level and thereby qualifying for 
the next level. Promotion to the next level takes place after an assessment has been made against 
the relevant competency profile and after other program requirements have been met. 

As demonstrated, the impact on the core and guiding values will depend on the circumstances 
surrounding the choice of a non-advertised appointment process. 
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Observations 


Overview 


While all four of the organizations examined had a formal human resources (HR) system, three of 
the four organizations did not rely solely on their HR system to provide their quarterly reports to the 
Public Service Commission (PSC). As a result, these organizations maintain a separate system — 
typically using Excel-based systems designed in-house — to manage this information. HR systems 
such as PeopleSoft, SAP and HRIS are used primarily for the storage of position information, and not 
for the management of the appointment process. The HR system is normally used to generate a 
process number at the beginning of the hiring process, while the remaining information would be 
captured subsequent to the appointment. 


Based on the file review of a random selection of files, almost half of all non-advertised processes 
are either acting appointments of four months or over, or are extensions of acting appointments. 
Table 1 lists, in order of frequency, the various types of non-advertised processes that the four 
organizations conducted during the 2007-2008 fiscal year. 


Table 1: Distribution of types of non-advertised processes 


Type of non-advertised processes # % 
Acting of four months or over GAG Na 36% 
Acting extensions 12% 
Student bridging 9% 
Developmental program — Organizational 7% 
Reclassification 8 6% 
Promotion 8 6% 
Incumbent-based appointment 8 6% 
Employment equity 7 5% 
Conversion of tenure of employment from term to indeterminate 6 4% g of 
Other 5 4% 
Shortage group 4 3% 
Special assignment pay plan 2 1% 
Developmental program — Government-wide 1 1% 
Total {137 100% 


Source: Audit, Evaluation and Studies Branch, Public Service Commission of Canada 
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As per the Appointment Policy Questions and Answers for the Choice of Appointment Process, 

the PSC has indicated a preference for advertised appointment processes to ensure respect of the 
euiding values. Respect of the value of access ensures that persons have a reasonable opportunity 

to apply and be considered for employment. However, under certain circumstances, non-advertised 
appointment processes provide the necessary flexibility to deputy heads. Appointments resulting 
from developmental programs, reclassifications, the special assignment pay plan (SAPP) and 
incumbent-based appointments, as well as some appointments of designated employment equity 
(EE) group members, will usually result in a non-advertised appointment process. For other types 
of appointments the use of non-advertised processes may have a greater impact on the guiding values. 


Relevance: The information gathered should meet the needs 
of the clients 


The relevance of statistical information reflects the degree to which it meets the needs of clients. 
Indicators ACC-1 and ACC-3 reflect the ratio of appointment processes that are non-advertised 
to external and internal appointment processes, respectively. The ratio of processes that are 
non-advertised is used as an indicator of access to federal government positions. High ratios 

of ACC-1 and ACC-3 may contribute to poor performance measures for organizations. 


There are common criteria for the use of non-advertised appointment processes among the 

four organizations. For example, all have included reclassification, acting for less than 12 months, 
developmental programs, student bridging and highly specialized skills as criteria for using a 
non-advertised appointment process. There are further criteria that are common among more than 
one organization, including SAPP, EE appointment, conversion of tenure of employment from term 
to indeterminate and emergency situations. Based on the above organizational criteria, the type of 
non-advertised appointment processes is captured electronically in three of the four organizations. 


Discussions with stakeholders within the PSC and the four selected organizations have raised 
concerns regarding the relevance of these indicators, given the manner in which they are calculated. 
Organizations have been critical of the fact that high ratios do not necessarily indicate reduced access 
to positions, given that non-advertised appointments may be used for promotions as part of a 
developmental program, when appointing a person to his/her reclassified position or when bridging 
a student into an indeterminate or term position. Organizations consider that these non-advertised 
processes do not represent a risk to the value of access. As a result, in February 2009, the PSC 

revised the type of appointment process applicable to student bridging appointments from 
non-advertised to advertised. 


Study on the data collection of non-advertised appointment processes 


As part of its own internal risk management strategy, the Canadian International Development 
Agency differentiates non-advertised processes into two separate categories: those considered 

to be of higher corporate risk (e.g. any appointments to EX-level positions, acting appointments over 
12 months or promotions) and those that present lower risk (e.g. reclassifications, promotions within 
an approved developmental program or student bridging). In addition to having separate approval 
processes for these two categories of non-advertised processes, the Canadian International Development 
Agency also produces separate statistics for each category, as they believe that it is more important 

to monitor and restrict the higher risk processes. 


The opinion that the current method of calculating ACC-1 and ACC-3 produces an ambiguous 
measure of an organization’s performance was expressed by all of the selected organizations. 

In addition, the PSC’s division responsible for handling organizational inquiries regarding the data 
analysis for leading indicators has noted that this opinion is expressed uniformly across government 
organizations. Similarly, the PSC’s division responsible for compiling the leading indicators and 
reporting them to organizations has expressed the opinion that more detailed information regarding 
the types of non-advertised processes would improve the relevance of the leading “Access” indicators. 
The ratios (ACC-1 and ACC-3) that are calculated by the PSC are discussed with the various 
organizations to determine whether or not they are relevant reflections of access to positions. 

This, as well as other information, is taken into account by the PSC in determining the final 

rating for these leading indicators. 


To illustrate the impact of redefining ACC-1 and ACC-3, the variables were recalculated using 

the data from the sample of files selected for the file review. Based on the current definition, 
ACC-1 (external non-advertised processes) for all four organizations combined is 24% and ACC-3 
(internal non-advertised processes) is 61%. If, for example, we calculate using only acting, acting 
extensions, promotions, conversion of tenure of employment from term to indeterminate 

(prior to the conversion period), shortage group and other appointments, the ratios become 

9% and 38%, respectively. 


Organizations were unanimous in their concern that the current method of calculating 
ACC-1 and ACC-3 leading indicators did not provide a relevant measure of access to public 


service positions. 
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Accuracy: The information gathered should correctly describe 
appointment information 


Accuracy is the degree to which information correctly describes the phenomena that it was designed 
to measure. Two elements of accuracy that are relevant for administrative data are coverage and 
validity. Coverage is the degree to which the list of appointments provided by organizations to the 
PSC is a true reflection of their population of appointments. Over-coverage occurs if the list of 
appointments includes appointments from outside the population. Under-coverage occurs if the list 
of appointments is missing appointments from within the population. Validity is the accuracy of the 
data found within the dataset. Both coverage and validity are necessary conditions for generating 
accurate indicators. 


The PSC receives organizations’ quarterly reports which list the characteristics of all specified 
appointments needed for calculating ACC-1 and ACC-3. These include appointments to the public 
service, promotions and acting appointments of four months or over. As a quality control procedure, 
the lists of appointments from the organizations are compared with a list of all known appointments 
from the PSC Fiscal Database, a database generated by the PSC from payroll information. The result 
of this comparison illustrates the degree of under-coverage within the quarterly reports. 


Overall, federal government organizations sent the PSC 69% of all appointments listed in the PSC 
Fiscal Database for the 2007-2008 fiscal year. Of the four organizations examined, Transport Canada 
provided a relatively complete list of appointments (87%). Library and Archives Canada and 
Citizenship and Immigration Canada had near-average under-coverage rates (68% and 67%), 

and the Canadian International Development Agency had the lowest under-coverage rate 

of the four selected organizations (57%). 


As part of the file review, the percentage of over-coverage was calculated. The quarterly reports sent 
from organizations to the PSC for the 2007-2008 fiscal year contained processes that should not 
have been included, such as deployments and term extensions. Only Transport Canada had an 
over-coverage rate of 0%, while the three remaining organizations had over-coverage rates that 
ranged from 7% to 12%. 


Of equal importance to coverage is the validity of the information that is sent to the PSC. The file 
review demonstrated that this information contained some errors. While the Canadian International 
Development Agency had the lowest coverage rate, we found no errors in the data they sent. 
Conversely, Transport Canada sent the most complete information; yet, it had the highest error rate 
(14%). It should be noted, however, that more than half of the errors discovered in Transport Canada 
files were likely due to a systemic problem, which has since been corrected. Some advertised processes 
were coded as non-advertised. Specifically, while the first appointment from an advertised process 
was coded as advertised, subsequent appointments under that same process number were coded 

as non-advertised. 
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Of the 40 files reviewed from the sample that organizations indicated were advertised, only 1 (2%) 
was actually non-advertised. Of the 152 files that organizations indicated were non-advertised, 

16 (11%) were found to be advertised. Six of these cases were subsequent appointments. 

This ambiguity between advertised and non-advertised appointments in the context of the collective 
staffing processes raised a number of questions in the HR community. It was clarified in January 2008 
in a Letter to Heads of HR (08-02) that, in order to maximize flexibility, managers can make advertised 
appointments from a pool if the advertisement includes information clearly indicating how, by whom 
and for what occupational groups and/or level(s) the results of the collective appointment process 
may be used. When appointments (i.e. the first and all subsequent appointments) made from the 
pool are consistent with the information provided in the advertisement, they are considered 
advertised appointments. 


Files were also assessed for their accuracy relative to being internal or external appointments. 
Of the 44 files that organizations listed as external appointments, 1 (2%) was actually internal and, 
of the 148 files that organizations listed as internal appointments, 7 (5%) were, in fact, external. 


Accuracy varies among organizations, given the existence of over-coverage and under-coverage 


as well as errors in coding. 


Timeliness: The information gathered should be available to clients in a 
reasonable amount of time 


Timeliness refers to the delay between the end of a reference period to which the information 
pertains and the date on which the information becomes available. In the case of the PSC quarterly 
reports, timeliness is affected by two stages of data processing. The first is the abstraction or entry 
of information into electronic format; the second is the extraction of this information into a format 
that can be used by the PSC, and any required quality control measures. 


Based on the interviews conducted with organizations, the delays associated with the entry of data 
into electronic formats is not a major factor affecting the timeliness of data. In the four organizations 
examined, information was typically entered into HR databases on a timely basis, with minimal backlogs. 


The delays associated with the extraction and preparation of data showed extremely wide variances. 
While Transport Canada could produce the PSC quarterly report in a matter of days and with 
minimal effort, Citizenship and Immigration Canada estimated that it required approximately one 
month of effort from 1.5 full-time equivalents to prepare one quarterly report. This level of effort 

is due in part to the fact that much of their HR data management is decentralized, a characteristic 
associated with the structure of the organization and the nature of the work they perform. 
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Library and Archives Canada and the Canadian International Development Agency also reported that 
the preparation of PSC quarterly reports requires a great deal of effort. Organizations are requested 
to submit their quarterly reports one month after the end of each quarter. In the first quarter of the 
2008-2009 fiscal year, approximately 15% of all federal government organizations were not able 

to meet this deadline. This statistic does not illustrate the differing levels of effort required by 
organizations to produce the PSC quarterly reports, but it does reveal that many organizations are 
not managing data in such a way as to easily access it. During the previous fiscal year, two of the four 
selected organizations asked for extensions for submitting their quarterly reports. 


The amount of time and effort required to submit data varies greatly between organizations. 
Fifteen percent of all federal organizations were not able to meet the one-month deadline for 


submitting quarterly reports. 


Accessibility: The information gathered should be easily accessed 


Accessibility refers to the ease with which statistical information can be obtained. Reduced 
accessibility of data can be associated with reduced accuracy, including poor coverage and validity. 
Data that are easily accessible are likely to be accessible through electronic means and to require 
minimal manipulation. The data is therefore accessed in a more standardised manner (i.e. through 
electronic queries). When errors occur in the performance of an electronic query, they tend to be 
systemic, that is, the same type of error will occur multiple times. Such an error requires only one 
correction to the HR system to eliminate the occurrence of that type of error. Alternatively, errors that 
are associated with manual manipulation can result from inexperience, fatigue, indifference or human 
error. As a result, the variance of this type of error is much greater and more difficult to detect 

and correct. 


As seen in the four organizations examined, there is an array of effort required to access information. 
In the case of the Canadian International Development Agency, information stored in SAP is 

not easily accessible given the amount of time and effort required to create customized queries 

(i.e. the Canadian International Development Agency estimates that the time necessary to adjust 

SAP to generate the PSC quarterly reports is approximately 18 months). As a result, the Canadian 
International Development Agency relies on an Excel-based system to manage the appointment 
process as a source of information for the PSC quarterly reports. As needed, the organization may 
refer to the appointment file to collect the required information. 


Bo sey on the data collection of non-advertised appointment processes 


At Citizenship and Immigration Canada, in order to help ensure data quality, production of the 
quarterly reports begins with reconciliation between data from their PeopleSoft system and an 
Excel-based system. Anomalies are then manually reviewed. 


Library and Archives Canada’s current version of HRIS does not allow for customized queries. 
Staff rely on the interpretation of the process number to determine the type of process or, in many 
cases, on a manual review of the original appointment file to gather all the data needed for the PSC 
quarterly reports. This organization is currently in the process of developing an Access database 

for managing appointment processes. 


Transport Canada is the only organization that uses its HR system to manage hiring process as well 
as position information. The organization has the capacity to easily modify its system and produce 
customized queries. The PSC quarterly reports are completed through electronic queries. 


The accessibility of data and the capacity to respond to changing requests for data varies 
greatly among organizations. One organization has the data stored centrally and has the 
ability to perform customized queries with little effort. The remaining organizations require 


more complex procedures to produce data, which significantly increase the risk of error 
and reduce timeliness. 


Interpretability: The meaning of the leading indicators should be 
easily understood 


Interpretability refers to the availability of supplemental information in order to properly understand 
the leading indicators. The organizations are responsible for providing data from which the PSC 
calculates the leading indicators. As such, it is the PSC’s responsibility to provide the supplemental 
information for interpreting the leading indicators. Both ACC-1 and ACC-3 are relatively simple 
variables, and our examination found sufficient internal PSC documentation explaining how the 
statistics are calculated. 


The calculation of the leading indicators related to access is easy to understand 


and is well documented. 
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Coherence: The information gathered by different organizations should 
be comparable 


Coherence of statistical information reflects the degree to which it can be successfully brought 
together with other statistical information. In this case, a high degree of coherence among the 
organizations in producing PSC reports is critical for being able to bring the data together and 
make valid inter-organizational comparisons. All organizations need to have standard concepts 
and classifications or groupings of data in order to produce reliable statistics. 


The 2006 Report on Preliminary Data Collection Activities Related to Advertised versus Non-advertised 
Appointments indicated that there were inconsistencies in the interpretation of certain appointment 
processes. As noted earlier, the PSC issued a letter in January 2008 to clarify when a collective staffing 
appointment should be reported as an advertised or a non-advertised appointment. 


However, a further data definition issue was identified following the file review of extensions to acting 
appointments that were originally of four months or over. One organization treated such extensions 
to acting appointments as subsequent non-advertised appointments, while the remaining organizations 
treated such extensions as administrative measures. However, for the purpose of the quarterly reports 
to the PSC, two organizations considered acting extensions as appointments. The Public Service 
Staffing Tribunal decisions indicate that extensions to acting appointments are considered appointments. 
However, there is currently no PSC policy interpretation on this matter. This data definition issue 
contributes to a reduced level of coherence of the data collected and reported by the PSC; further, 

it can affect inter-organizational comparisons, as well as inflate the ratio of non-advertised 
appointments for certain organizations. The PSC will examine this issue as part of its policy 

review and its five-year review of the PSEA. 


The data definition issue surrounding extensions to acting appointments reduces the level 


of coherence of information reported to the PSC. 
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Concluding remarks 


The quality of data from organizations depends on several interrelated factors, including the size 
and structure of the organization, the type of human resources (HR) system being used and the 
availability of resources to maintain HR information. This study illustrates how the challenges to 
producing quality data vary greatly among organizations. 


Transport Canada has a system that can be used to manage appointments and easily produce data. 
Although data is entered in several regions, it is managed centrally. This system can be modified 
to meet Transport Canada’s needs and those of central agencies. 


The Canadian International Development Agency’s current HR system, SAP, is used in few other 
organizations. There are few resources available with the capacity to modify the system, 
limiting its responsiveness. 


Library and Archives Canada is a medium organization which has limited resources available 
for maintaining a complex HR system. 


The management of Citizenship and Immigration Canada’s HR data is decentralized and must 
be cross-referenced to a secondary tracking system, contributing to the difficulty of reporting 
information regarding select appointments. 


In three of the four organizations examined, detailed information regarding non-advertised processes 
is captured electronically but, for two, it remains difficult to access. If the data were accessible, a more 
relevant measure of access to positions could be calculated. Organizations, however, already seem 

to have great difficulty meeting the current demands of the Public Service Commission (PSC) 

quarterly reports. 


The PSC is committed to clarifying the policy interpretation surrounding extensions to acting 
appointments. Further, the PSC will explore other options and methods of data collection 
from organizations. 
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Appendices 


Appendix 1 — About the study 
Objective 


The purpose of the study was to describe data collection practices related to non-advertised appointment 
processes and the limitations of the data available, as well as to explore associated risks and areas for 
improvement for the Public Service Commission (PSC) and organizations. 


Approach 


The study focused on the non-advertised appointment activities of four organizations for the period 
of April 1, 2007, to March 31, 2008. Organizations were selected based on the following criteria: 

(1) obtaining a critical mass of non-advertised appointment processes; (2) obtaining organizations 
that use different human resources (HR) information management systems; (3) obtaining a mix of 
medium and large-sized organizations; and (4) obtaining both centralized and decentralized organizations. 
As a result of the above selection process, the following organizations were included in the study: 

m the Canadian International Development Agency; 

m Citizenship and Immigration Canada; 


m Library and Archives Canada; and 


m= Transport Canada. 


Multiple lines of evidence were used to address the study objectives, including the following: 


= Interviews with key stakeholders, including the following representatives within each 
organization: HR operations, HR oversight and HR systems specialists; 


m= Examination of the organizational information of appointment processes and its potential 
use for data analysis; 


m Observation (walk-through) of the data collection process from the appointment file to the HR 
information management system; 


= Documentation review, including various monitoring reports and criteria for non-advertised 


appointment processes; and 


m File review of a representative sample of appointment processes. 


Study on the data collection of non-advertised appointment processes cl 


Sampling method 


The quarterly reports of all appointments from the 2007-2008 fiscal year for the four selected 
organizations were used as the sampling frame for the selection of processes for file review. 
Forty non-advertised processes and ten advertised processes were randomly selected from each 
organization, with the exception of Library and Archives Canada, who had only thirty-six 
non-advertised appointment processes during that fiscal year. Files that were out of scope, 
such as deployments or term extensions, were replaced with other randomly selected files. 
Table 2 summarizes the populations, sample sizes and number of files found out of scope 

for each organization. 


Table 2: Summary of sampling 


Organization Advertised Non-advertised Advertised Non-advertised 

Canadian 

International 

Development 

Agency 2H 332 10 40 7 
Citizenship 

and Immigration 

Canada 706 680 10 4 
Library and 

Archives Canada 188 36 | 10 32 5 
Transport 

Canada 704 674 10 40 0 
Total 1 869 1 722 40 152 16 


Source: Audit, Evaluation and Studies Branch, Public Service Commission of Canada 


Confidence intervals from samples of non-advertised appointment processes for individual 
organizations are +/-10% or better, given a 90% confidence level and deviation rates of 20% or less. 
The confidence interval from the sample of non-advertised appointment files for all organizations 
combined is approximately +/- 5%, given a 90% confidence level and deviation rates of 20% 

or less. The confidence interval from the sample of advertised appointment files for all organizations 
combined is approximately +/- 10%, given a 90% confidence level and deviation rates of 20% or less. 
When necessary, weighted ratios are calculated to account for non-proportional sampling. 
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Appendix 2 — Reporting methodology 


The study made use of the Statistics Canada’s Quality Assurance Framework to define data quality 


and organize the lines of inquiry for the study and the report findings. This framework defines six 
dimensions of data quality. 


Relevance The relevance of statistical information reflects the degree to which it meets 
the real needs of clients. It is concerned with whether the available information 
sheds light on the issues of most importance to users. Assessing relevance is a 
subjective matter, dependant upon the varying needs of users. The Agency’s 
challenge is to weigh and balance the conflicting needs of current and potential 
users to produce a program that goes as far as possible in satisfying the most 
important needs, given resource constraints. 


Accuracy The accuracy of statistical information is the degree to which the information 
correctly describes the phenomena that it was designed to measure. It is 
usually characterized in terms of error in statistical estimates and is 
traditionally decomposed into bias (systemic error) and variance (random 
error) components. It may also be described in terms of the major sources of 
error that potentially cause inaccuracy (e.g. coverage, sampling, non-response 
or response). 


Timeliness The timeliness of statistical information refers to the delay between the 
reference point (or the end of the reference period) to which the information 
pertains and the date on which the information becomes available. It is typically 
involved in a trade-off against accuracy. The timeliness of information will 


influence its relevance. 


Accessibility The accessibility of statistical information refers to the ease with which it can 
be obtained from the Agency. This includes the ease with which the existence 
of information can be ascertained, as well as the suitability of the form or 
medium through which the information can be accessed. The cost of the 
information may also be an aspect of accessibility for some users. 


The interpretability of statistical information reflects the availability of the 
supplementary information and metadata necessary to interpret and utilize 
it appropriately. This information normally covers the underlying concepts, 
variables and classifications used, the methodology of data collection and 
processing and indications of the accuracy of the statistical information. 


Interpretability 


The coherence of statistical information reflects the degree to which it can be 
successfully brought together with other statistical information within a broad 
analytic framework and over time. The use of standard concepts, classifications 
and target populations promotes coherence, as does the use of common 
methodology across surveys. Coherence does not necessarily imply full 


Coherence 


numerical consistency. 


Source: Statistics Canada’s Quality Assurance Framework, September 2002 
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Appendix 3 — Leading indicators of long-term success 
Intended effects: Progress in improved short-term outcomes 
Results: Effectiveness and adherence to staffing values 


Merit 
MER-1: Managers’ satisfaction with the quality of hires; 


MER-2: Percentage of non-imperative appointments where the person does not meet the official 
language profile within the time periods prescribed by regulations; 


MER-3: Candidates’ perceptions on whether actual job requirements match advertised requirements; 


MER-4: Candidates’ perceptions on whether the posted qualifications and criteria for positions are 
bias-free and barrier-free; 


MER-5: Trends in founded Public Service Staffing Tribunal complaints and Public Service 
Commission investigations into staffing; and 


MER-6: Trends in in-house investigations and founded complaints. 
Non-Partisanship 

NP-1: Managers’ perception of external pressure to select a particular candidate; and 
NP-2: Number of founded complaints related to political influence in staffing. 


Representativeness 


REP- 1: Staffing strategies — employment equity (EE) [distribution of hires versus work-force 
availability]; and 


REP-2: Number and percentage of internal and external staffing advertisements with EE provisions 
— to enhance access of EE group members to public service positions. 


Access 

ACC-1: Ratio of non-advertised external processes to total; 
ACC-2: Ratio of external processes advertised nationally to total; 
ACC-3: Ratio of non-advertised internal processes to total; and 


ACC-4: Trends of managers’ selection of staffing process. 
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Fairness 


FAIR-1: Candidates’ perceptions of the fairness of the assessment process; and 


PAIR-2: Percentage of acting appointments that becomes indeterminate in the same or similar 
position in the same organization. 


Transparency 


TRANS-1: HR plans and staffing strategies are communicated on organizations’ Web sites, 
and contents are clearly communicated to employees and managers; 


TRANS-2: Organizations demonstrate consultation and communication of staffing-related 
information to unions; 


TRANS-3: Length of time that job openings are posted; and 


TRANS-4: Candidates’ perceptions of openness and transparency in internal staffing. 
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Appendix 4 — Glossary 


Access 

One of the guiding values of the Public Service Employment Act requiring that persons from across 
the country have a reasonable opportunity, in their official language of choice, to apply and to be 
considered for a public service employment. 


Acting appointment 
The temporary promotion of an employee. 


Acting extension 
The increase in the length of the temporary performance of the duties of another position by 
an employee. 


Advertised appointment process 
When persons in the area of selection are informed of and can apply to an appointment opportunity. 


Appointment 
An action taken under the Public Service Employment Act to hire or promote someone. 


Appointment values 
Core values (merit and non-partisanship) and guiding values (fairness, access, transparency 
and representativeness). 


Collective staffing process 
An approach that allows for one staffing process to fill several similar positions within or between 
departments and agencies, as opposed to several individual processes. 


Confidence interval 


An interval which has a known and controlled probability (generally 95% or 99%) to contain the 
true value. 


Conversion of tenure of employment from term conversion 

When a person has been employed in the same organization as a term employee for a cumulative 
working period of three years without a break in service longer than sixty consecutive calendar days, 
the organization must appoint the employee indeterminately at the level of his/her substantive position. 


Deployment 

The movement of a person from one position to another in accordance with Part 3 of the Public 
Service Employment Act. A deployment cannot be a promotion and cannot change the tenure of 
employment from specified term to indeterminate. A person who is deployed is no longer the 
incumbent of his or her previous position. 
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Designated groups 
As defined by the Employment Equity Act, women, Aboriginal peoples, persons with disabilities 
and members of visible minorities. Often referred to as employment equity designated groups. 


Developmental programs 


Programs that offer learning and leadership development by providing hands-on work assignments 
and formal learning components. 


Deviation rate 

The total variation displayed by a set of observations, as measured by the sums of squares 

of deviations from the mean, may in certain circumstances be separated into components 
associated with defined sources of variation used as criteria of classification for the observations. 


External appointment process 
A process in which persons may be considered, whether or not they are employed 
in the public service. 


Fairness 

One of the guiding values of the Public Service Employment Act requiring that decisions be made 
objectively and free from political influence or personal favouritism, that policies and practices reflect 
the just treatment of persons, and that persons have the right to be assessed in their official language 
of choice. 


Incumbent-based appointment 
An internal appointment process within the Research and University Teaching groups, if there 
is a career progression framework established by the deputy head. 


Indeterminate employment 
Employment of no fixed duration, whether part-time, full-time or seasonal. 


Internal appointment process 
A process for making one or more appointments for which only persons employed in the public 
service may be considered. 


Leading indicators 

Infrastructure, processes, practices and actions, as articulated under the Staffing Management 
Accountability Framework’s Key Change Areas and associated indicators, that are necessary 
for the achievement of positive staffing results. 


Merit 

One of the core values of the Public Service Employment Act. An appointment is made on the 

basis of merit when a person appointed meets the essential qualifications for work to be performed, 
as established by the deputy head, including official language proficiency. The sub-delegated manager 
may also take into account any current or future asset qualifications, operational requirements and 


organizational needs. 


Study on the data collection of non-advertised appointment processes 


Non-advertised appointment process 
An appointment process that does not meet the criteria for an advertised appointment process. 


Non-partisanship 

One of the core values of the Public Service Employment Act, non-partisanship ensures that 
appointments and promotions to and within the public service are made free from political influence 
and supports the capacity and willingness of employees to serve governments, regardless of 

political affiliation. 


Over-coverage 
Arises from the presence in the frame of units not belonging to the target population and of units 
belonging to the target population that appear in the frame more than once. 


Promotion 
An appointment to a higher-level position. 


Public service 

As defined by the Public Service Employment Act, the organizations named in Schedule I to the 
Financial Administration Act, the organizations named in Schedule IV to that Act and the separate 
agencies named in Schedule V to that Act. 


Reclassification 

The reclassification of a position occurs when the evaluation of its work description results in a 
change to either or both its occupational group and level, due to a significant change in the work 
assigned to it. A reclassification may be upward, if the work becomes more demanding, or 
downward, if the work becomes less demanding. 


Representativeness 

One of the guiding values of the Public Service Employment Act requiring that appointment processes 
be conducted without bias and without creating systemic barriers in order to help achieve 

a public service that reflects the Canadian population it serves. 


Shortage group 
Lack of persons available having the skills, abilities or training required for the functional 
community. 


Special assignment pay plan 

Organizations may assign a specified number of employees, for a period of up to three years, to 
duties and responsibilities that have not been classified or that have been classified at a lower level 
to which the employee is appointed. 


Specified term employment 
Part-time or full-time employment of a fixed duration. 


js Study on the data collection of non-advertised appointment processes 


Staffing activities 


Include internal and external appointments to the public service, deployments, 
assignments and secondments. 


Student bridging 

A mechanism that allows managers to hire qualified recent post-secondary graduates who have 
previously worked in the public service through student employment programs, such as the 
Federal Student Work Experience Program, the Co-operative Education and Internship Program 
or the Research Affiliate Program, in accordance with the organizational criteria or policies. 


Term extension 
The increase in the length of the employment of a fixed duration 


Term to indeterminate 
The non-advertised appointment of the employee indeterminately at the level of the employee’s 
substantive position prior to the term conversion period. 


Transparency 
One of the guiding values of the Public Service Employment Act, requiring that information 
about strategies, decisions, policies and practices be communicated in an open and timely manner. 


Under-coverage 
Results from the omission from the frame of units belonging to the target population. 
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